The subject matter disclosed herein generally relates to test case reduction and, more particularly, to test case reduction in ABI compatibility testing.
Many operating systems include an Application Binary Interface (ABI) that defines the interface between a calling function and the function that is the target of the call. For example, the ABI may define when arguments are passed in memory versus in registers for particular data types or data sizes. Errors can occur when a compiler is ported to a new operating system and therefore needs to support a new ABI, or when a new compiler is created for an existing operating system.
To ensure proper testing of the ABI a number of test cases can be written or automatically generated. The test cases are run to test and ensure that all parameters and return arguments are passed correctly between the calling and called functions and can include a large variety of parameters and return types. When an error is detected, it is often in a function containing a large number of randomized arguments, and it can be a challenging and manual process to iteratively remove arguments from the called function (on both the caller and callee side) until the minimal set of arguments that cause a failure are found. Without this minimal set, it is very difficult for the developer to diagnose what aspect of the ABI the compiler has failed to correctly implement.
Further, current techniques for reducing the set of arguments to those that produce the error are time consuming because the removal process is cumbersome and the removal may change the underlying logic causing detection errors. Specifically, when an argument is removed from the function, it is removed from at least three places: the function call in the caller, the function definition in the callee, and the function declaration in the header file. Additionally, when an argument is removed, the logic of the function itself may change, and manual steps may be required to remove all uses of that argument from the called function. Further, once all instances of the argument are removed from the called function, the logic of the function may change and the ability of the function to validate that parameters were passed correctly is compromised, requiring further manual work to correct any errors.
Accordingly, test case reduction in ABI compatibility testing is desired through a method and system of determining a minimal set of arguments that cause the event that the test case is testing for.